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THE  EFFECT  OF  AN  INTERFERING  SIGNAL 
ON  THE  PERFORMANCE  OF  NONPARAMETRIC  DETECTORS 
USING  SPECTRAL  ESTIMATES 

by 

M.  J.  Wilmut  and  R.  F.  MacKinnon 

ABSTRACT 

-V 

Nonparametric  statistics  have  been  previously  applied  to  spectral 
data  in  order  to  detect  narrow  band  signals.  Here  we  show  the  degradation  in 
performance  due  to  the  presence  of  an  interfering  signal.  A modified  decision 
scheme  is  proposed  to  overcome  this  problem.  It  is  shown  that  with  this  new 
technique  the  presence  of  even  a strong  interfering  signal  does  not  seriously 
affect  the  ability  to  detect  signals. 

\ 
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INTRODUCTION 

We  consider  the  situation  in  which  signals  embedded  in  a time  se- 
ries appear  as  narrow  lines  in  the  frequency  domain.  Detection  is  based  on  the 
matrix  of  spectral  estimates  : i=l,2,...,M;  j=l,2,.  . . ,N.  Here  time  is 
referenced  by  the  i parameter  and  frequency  by  the  j parameter.  The  parameter 
N is  the  largest  number  of  frequency  estimates  for  which  the  spectrum  can  be 
said  to  be  flat,  while  M is  the  number  of  spectral  estimates  per  frequency 
cell  to  be  used  in  the  decision  process. 

Let  R„  be  the  ordered  rank  of  the  for  a fixed  time  interval, 

that  is  i fixed,  j=l,2,...,N.  The  null  hypothesis  is  that  all  N frequencies 
contain  noise  only.  For  the  X„  independent  identically  distributed  random 
variables,  the  R^  are  uniformly  distributed  over  the  integers  1,2,...,N.  An 
alternative  hypothesis  is  that  a signal  occurs  at  at  most  two  of  the  frequencies 
represented.  In  order  to  determine  the  distribution  of  the  ranks  in  this  case, 
one  must  assume  a specific  distribution  for  the  X^ ; that  is,  one  must  specify 
the  nature  of  the  signal  and  the  noise. 

Various  authors  (1),  (2),  (3)  have  considered  the  simple  alternate 
hypothesis  that  a signal  occurs  at  one  frequency.  In  the  following  we  analyze 
the  performance  of  the  nonparametric  decision  scheme,  which  gave  good  results 
for  the  one  signal  alternative  (3),  when  two  signals  are  present.  It  is  seen 
that  the  probability  of  detection  can  be  seriously  degraded  by  an  interfering 
signal.  Finally  a modified  detector  is  proposed  and  analyzed. 

DISTRIBUTION  OF  THE  RANK  STATISTICS 

Suppose  X^, . . . ,X^  represent  a set  of  N independent  variable  spec- 
tral estimates  of  which  the  first  N-2  correspond  to  frequencies  without  signal 
components  (called  noise  frequencies),  X^  ^ and  X^  correspond  to  cells  contain- 
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ing  signals.  (We  call  cell  N-l  the  interfering  signal  frequency  and  cell  N 
the  signal  frequency).  We  examine  the  case  where  the  N estimates  are  expon- 
ential random  variables;  the  first  N-2  estimates  have  mean  tp , ^ has  mean 

1(1(1+62)  and  X^  has  mean  iJ>(1+0^),  where  0^  and  8^  are  the  respective  cell  sig- 
nal-to-noise  ratios  and  are  hence  non-negative. 

We  wish  to  find  P(k)  the  probability  that  signal  takes  rank  k 
where  k=l,2,...,N.  This  occurs  if  either 

a)  the  interfering  signal  estimate  and  k-2  noise  frequency 
estimates  are  less  than  X^  and  N-k  noise  frequency  estimates 
are  greater  than  X^  or 

b)  k-1  noise  estimates  are  less  than  X^  with  the  remaining  N-k-1 
noise  frequencies  and  interfering  signal  estimate  greater 
than  Xjj. 

We  find  the  probability  of  one  specific  ordering  of  (a)  and  (b) 
and  then  multiply  by  the  number  of  ways  these  can  occur.  Now  since  X , ...,Xjj 
are  independent,  the  probability  of  one  specific  ordering  for  (a)  is 


dx 


where  P denotes  the  probability  of  an  event  and  f(x)  is  the  density  function 
of  X^.  The  probability  of  one  specific  ordering  of  event  (a)  is  given  by 


( 

k-2  specified  noise  esti- 

remaining N-k  noise 

1 f(x)P 

mates  and  interfering 
signal  estimate  are  less 
than  x 

P 

estimates  are  greater 
than  x 

Wi+ej  exp  I" 


2-J/(l+01) 


Ik-  21 


!-exp(- 2^~) 


1““p("  2Mi+0o>Txp("2n 


TN-k 


dx 


“l  / [1_exP(_y)]  k~Z  [l-exp(-c*2y)]  expC-a^y)  [exp  (-y)] 


N-k 
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where  a^d+e^-1,  c^-U+Oj)”1,  y= 


I |l-exp(-y)Jk  2 exp (-y (N-k+c^) ) dy  -Vjl-exp(-y)J  k 2exp(-y  (N-k+c^+c^)  )dy 


~0  0 
which  by (4)  page  305  Formula  3.312-1  is 

= a1J^B(N-k+a1,k-l)  - BCN-k+cij+c^,  k-l)J  , 

or  replacing  B(x,y)  the  Beta  function  with  the  equivalent  Gamma  function, 


r(N-k+a;L)  T(k-l)  r(N-k+ai+a2)  r(k-l) 


i rdw-^-i) 


r(N+ai+«2-i) 


Now  this  situation  can  occur  in 
is 


(“) 


ways.  Hence  the  probability  of  event  (a) 


a^f(N-l)  T(N-k+a^)  rCN-k+a^+o^) 


r(N-k+i)  r(N+ai-D  r(N+u1+a2-i) 


The  probability  of  any  specific  ordering  of  event  (b)  is  given  by 

/%  00 

1 2V(l+0  ) 6XP  (-x/2,<'(1+ei^[1-exP(_x/2^)]k  1[exp(-x/2ij;)] N k 1jexp(-x/20(l+92))]dx 
= ax  j* |l-exp(-y)|k  1 exp(-a;ly)  jexp(-y)JN_k_1  exp(-a,y)  dy 


where  and  are  as  defined  above  and  y*x/2i|i 


00 

( k-l 

Oj  I (1-exp (-y))  exp(-y(N-k-l+a1+a2>)  dy 

Jo 

a.B(k,N4n  +a  -k-i)  - ^rWrt^^^k-D 

F(N+a  +a9~l) 

/ N-2\ 

jw  this  specific  event  can  occur  in  I ^ ^ J ways. 


r(N+aj+ct2-d 


Now  this  specific  event  can  occur  in 


Hence  the  probability  of 
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event  (b)  is 


a1T(N-l)  r(N+ct1+a2-k-l) 
T(N-k)  r(N+«1+a2-i) 


Thus 

P(k)=P(k,N,«1>a2)  = 


air(N-i) 

r(N-k+i) 


r(N-k+a1> 

T(N4a  -1) 


^al+a2-1^ 


r(N-k+a1+a2-l) 

r(N+ai+a2-l) 


(1) 


DETECTOR  PERFORMANCE 

The  decision  scheme  is  based  on  the  sum  of  M independent  random 
variables  representing  some  function  of  the  rank  at  each  frequency.  That  is, 
we  decide  a signal  is  present  at  frequency  j depending  on  the  value  of 

M 

Vh  a(Rij} 

where  a(x)  is  some  function  of  x.  The  statistics  of  T^  can  be  found  for  any 

function  a(x),  for  small  M we  use  numerical  convolution  and  for  large  M the 

Central  Limit  Theorem.  We  study  below  the  function  which  gave  the  best  results 

for  the  one  signal  alternate  hypothesis  (3),  the  Savage  statistic.  Here 
N 

a(x)  = i_1 

J.-N-X+1 

Computer  programmes  were  written  to  determine  the  performance  of 
the  Savage  statistic  for  various  values  of  M,  N,  0^  and  ©2.  The  solid  lines 
in  figures  1 to  4 give  typical  results  for  the  original  procedure. 


PROBABILITY  OF  DETECTION 
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SIGNAL  2 CELL  SIGNAL  TO  NOISE  RATIO  (<JB) 


Plgure  1.  Signal  2 cell  slgnal-to-nolse  ratio  versus  probability 
of  detection  for  signal  1 and  for  signal  2.  False 
alarm  rate  10  ; A 0 accumulants , 16  frequencies  ranked, 


signal  1 cell  slgnal-to-nolse  ratio-'idB. 
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“I 1 1 1 r 
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SIGNAL 2 CELL  SIGNAL  TO  NOISE  RATIO  (dB) 


Figure  2.  Signal  2 cell  slgnal-to-nolse  ratio  versus  probability 

_2 

of  detection  for  signal  1.  False  alarm  rate  10  ; 

60  accumulants , 8 frequencies  ranked,  signal  1 cell 
slgnal-to-nolse  ratio-2dB. 
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Figure  3.  Signal  2 cell  slgnal-to-nolse  ratio  versus  probability 

_2 

of  detection  for  signal  1.  False  alarm  rate  10  ; 

*10  accumulants,  16  frequencies  ranked,  signal  1 cell 
slgnal-to-nolse  ratio-ildB. 
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SIGNAL  2 CELL  SIGNAL  TO  NOISE  RATIO  (dB) 

« 

Figure  *4.  Signal  2 cell  slgnal-to-nolse  ratio  versus  probability 

_? 

of  detection  for  signal  1.  False  alarm  rate  10  ; 

100  accumulants,  32  frequencies  ranked,  signal  1 cell 
slgnal-to-nolse  ratio-6dB. 
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When  the  interfering  signal  power  is  low  compared  to  that  of  the  signal,  the 
interfering  signal  does  not  have  much  effect  on  the  probability  of  detection. 
However,  when  the  interfering  signal  power  is  much  greater  than  that  of  the  sig- 
nal, detection  probabilities  are  greatly  affected. 


MODIFIED  DETECTOR  PERFORMANCE 

In  order  to  improve  performance  when  two  or  more  signals  are 
present  we  propose  the  following  scheme.  It  is  assumed  that  the  signal  of 
interest  is  present  for  at  least  twice  the  time  necessary  to  accumulate  the 
statistic  T_. . We  find  bounds  on  the  probability  of  detection  for  the  second 
and  later  time  intervals. 

A frequency  j will  be  called  significant  if  T^  is  greater  than 
some  constant  K.  The  constant  K is  determined  so  that  the  detector  has  a pre- 
assigned false  alarm  rate  a.  Of  course  a significant  frequency  could  be  either 
a false  alarm  or  a true  detection. 

Modified  Detector 

STEP  1 : Let  ft(l) , . . . ,£(L)  be  the  set  of  L significant  frequencies  of  the 

previous  decision.  (In  cases  of  practical  interest  L will  be 
small).  There  will  then  be  N-L  non-significant  frequencies.  Call 
this  set  NF. 

STEP  2(a):  The  decision  whether  frequency  £(i)  is  significant  is  made  by 
determining  the  rank  statistic  of  frequency  £(i)  when  t.(i)  and 
the  N-L  frequencies  in  the  set  NF  are  ranked.  Suppose  L^  such 
lines  are  significant  (L^  £ L) . 

STEP  2(b)  The  decision  whether  a frequency  in  the  set  NF  is  significant  is 
based  on  the  rank  statistic  of  that  frequency  when  the  N-L  fre- 
quencies of  that  set  are  ranked.  Suppose  L ^ such  frequencies  are 
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significant  (L2  N-L) . The  false  alarm  rate  is  a for  all  decisions. 
STEP  3:  Hence  a total  of  L-j^l^  lines  are  significant  and  is  the 

value  of  L for  the  next  decision  interval. 

In  practice  we  would  choose  N so  that  is  small.  The  follow- 

ing theoretical  analysis  keeps  the  two  most  significant  lines  if  L^+L?>3  and 
assumes  at  most  two  frequencies  contain  signals. 

Detector  Performance 

Let  P(0^,02>N)  be  the  probability  of  detecting  a sivnal  of  signal- 
to-noise  ratio  0^  in  the  presence  of  an  interfering  signal  of  noise  power  0^ 
when  N frequencies  are  ranked.  Similarly  PCS^.O^.N)  is  the  probability  of 
detecting  a signal  of  signal-to-noise  ratio  6^  when  we  have  an  interfering 
signal  of  signal-to-noise  ratio  0^.  We  assume  some  value  of  M is  given. 

Let  P(S^)  be  the  overall  probability  of  detecting  signal  1.  Then 
P(S^)  = P(0^’02’N)  f°r  the  original  procedure. 

In  order  to  determine  the  performance  of  the  modified  system  we 
must  find  the  false  alarm  and  detection  probabilities  under  the  various  hypo- 
theses and  compare  these  results  with  the  original  scheme.  The  comparison  is 
made  easier  by  the  fact  that  for  the  modified  procedure  we  keep  the  false 
alarm  rate  at  a constant  a for  all  decisions.  Hence  we  need  only  determine 
the  probability  of  detection  for  the  modified  procedure  for  the  alternate 
hypothesis . 

Suppose  L=0  and  two  signals  and  S2  are  now  present  with  0^  and 
©2  their  respective  signal-to-noise  ratios.  For  the  f irst  decision  interval 


we  have 
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EVENT  1 L=0 

p(sJevent  l)  = p(e1,e2,N) 
p(s2|event  1)  = P(01,02,N) 

At  the  end  of  this  interval  L can  be  0,  1 or  2.  We  list  the  possibilities  when 
L is  1 or  2 and  what  this  means  for  the  following  decision  interval. 

EVENT  2 L=1  and  S^  found  previously 

PCsJ  EVENT  2)  = p(e1>02,N) 

P(S?| EVENT  2)  = P(O,02,N-1) 

EVENT  3 L=1  and  S2  found  previously 

P(S1|EVENT  3)  = P(01,O,N-1) 

P(S2| EVENT  3)  = P(01,02,N) 

EVENT  4 L=1  and  a false  alarm  found  previously 

P(s| EVENT  4)  = P(0  ,02,N-1) 

P(S2|eVENT  4)  = P(01,02,N-1) 

EVENT  5 L=2  and  S^  and  S2  found  previously 

P ( S 1 | EVENT  5)  = P(0  ,0,N-1) 

P(S2| EVENT  5)  = P(O,0  ,N-1) 

EVENT  6 L=2  and  S^  and  a false  alarm  found  previously 

P(S1| EVENT  6)  = P(01,02>N-1) 

P(S  [ EVENT  6)  = P(O,0  ,N-2) 

EVENT  7 L=2  and  S2  and  a false  alarm  found  previously 

P ( S x | EVENT  7)  = P(@1,0,N-2) 

P(S  | EVENT  7)  = P(0  ,02,N-1) 

EVENT  8 L=2  and  two  false  alarms  found  previously 

P(S1I EVENT  8)  = P(01,02,N-2) 

P ( S 2 1 EVENT  8)  = P(01,02,N-2) 
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The  above  eight  events  are  mutually  exclusive  and  their  union 
covers  the  sample  space. 

For  the  first  interval  as  L=0  by  assumption 

P(s1)  = p(s1|event  1)  = p(61,92,n) 

For  the  second  interval 

P(S1)  = P( EVENTS  3,  5 or  7 occur).  P(S1|EVENTS  3,  5 or  7 occur) 
+ (1-P (EVENTS  3,  5 or  7 occur))  P(S  |NOT  EVENTS  3,  5 or  7 occur) 
Now  P (EVENTS  3,  5 or  7 occur)  = P(S2  detected  previously) 

= P(91,02,N) 

Next 

Min(P(0^,O,N-2) , P(0^,O,N-1))  _<  P (S^ | S 2 detected  previously) 

£ max  (P(01,O,N-2) , P(6  ,0,N-1). 

In  all  calculations  performed  using  both  the  Mann-Whitney  and  Savage  statistics 
we  found  P(0^,02>N-2)  <_  P(0^,02>N-1) . In  other  words  increasing  the  number  of 
reference  noise  spectrum  estimates  ranked  increases  the  probability  of  detect- 
ing signal  1.  However  this  result  could  not  be  proved  analytically.  This 
result  does  not  hold  for  all  functions  a(x). 

Thus  we  may  say 

P(0^,O,N-2)  £P(S^|S2  detected  previously)<P(0^,O,N-l) 

P(01’92’N~^)— p ( S 1 1 S 2 not  detected  previously)£P(0^,02 ,N) 

We  derive  an  upper  and  lower  bound  on  P(S^)  for  the  second  decision 
interval  (assuming  the  two  signals  are  still  present  during  this  time). 


P(S1)>P(01,02,N)P(01,O,N-2)  + 

(l-P(01,02,N))P(01,02,N-2) 

(2a) 

P(S1)<P(01,02,N)P(01,O,N-1)  + 

(1-P(01,02,N))P(01,02,N) 

(2b) 

Similar  inequalities  hold  for  P(S2). 
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Recursion  relations  can  be  derived  for  P(S^)  and  PCS^)  for  the 
third  and  later  intervals.  We  note  some  features  of  these  bounds.  First  if 
02  is  infinite  and  9^  finite,  P(9^,“,N)  = 1 and  we  achieve  the  bounds 
P(0^,O,N-2)£P(S^)<_P(0^,O,N-1)  for  the  second  interval  as  compared  to  P(S^) 
=P(0^,°°,N)  for  the  original  procedure.  Also  it  will  be  seen  that  for  N large 
the  difference  between  the  upper  and  lower  bounds  for  P(S^)i=l,2  is  almost 
zero. 

Figures  2,  3 and  4 give  typical  results  using  the  original  and 
modified  procedures.  Figure  2 shows  a case  for  low  N (N=8) . We  note  the 
upper  and  lower  bounds  especially  for  small  values  of  0,  are  different. 

Figures  3 and  4 illustrate  results  when  N = 16  and  32.  Here  only 
the  lower  bound  is  given,  the  upper  bound  is  very  close  to  it. 

In  all  cases  there  is  a degradation  in  the  probability  of  detec- 
tion using  the  modified  procedure  (as  compared  to  the  original  procedure)  when 
the  interfering  signal  power  is  low.  This  undesirable  effect  is  balanced  by 
the  large  improvement  in  detection  capabilities  when  the  interfering  signal 
power  is  large. 

CONCLUSIONS 

We  have  determined  the  effect  on  a nonparametric  detection  scheme 
of  an  interfering  signal.  If  the  strength  of  this  signal  is  large,  signal 
detection  probability  can  be  greatly  reduced. 

A modified  decision  procedure  was  given  and  lower  and  upper  bounds 
on  its  detection  capabilities  derived.  For  interfering  signals  of  low  strength 
the  modified  procedure  may  be  slightly  worse  than  the  original  whereas  for 
interfering  signals  of  high  signal-to-noise  ratios  the  modified  procedure  gives 
much  better  results  than  the  original  scheme. 
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Thus  using  the  modified  procedure  the  presence  of  a strong  inter- 
fering signal  does  not  seriously  affect  the  detectability  of  a weak  signal. 


'Ms 
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